Technologically Assisted Medical Systematic Reviews

In this demo, we present an extension to a variable threshold approach to tackle the problem of a systematic review task based on the active learning framework proposed by (Di Nunzio et al. 2017; Di Nunzio 2018). This approach defines a stopping strategy based on the geometry of the two-dimensional space of documents (Di Nunzio 2014; Di Nunzio 2017) that uses the explicit relevance feedback information given by the expert to automatically estimate the number of documents (medical publications or clinical trials) that need to be read. This approach has demonstrated to outperform all the other systems by reaching a recall recall greater than 95% with 25,000 documents less than the best performing systems.

Demo

The demo has been implemented using the Shiny Dashboard R package (https://rstudio.github.io/shinydashboard/). The application has two main parts: a sidebar for the main interaction with the system and the body for monitoring the results in real time.

Main Dashboard

The main dashboard shows the pieces of information we need in order to estimate the costs of the systematic review and the amount of relevant information found which is updated every time a clinician reviews a document.

Analysis Panel

The second panel shows a detailed analysis on the probability distribution of the relevant documents in the collection as well as two word clouds that show the most frequent terms found in the whole collection and in the relevant documents

References

Di Nunzio, G. M. 2014. “A New Decision to Take for Cost-Sensitive Naïve Bayes Classifiers.” Inf. Proc. & Manag. 50 (5): 653–74. doi:10.1016/j.ipm.2014.04.008.

Di Nunzio, G. M. 2017. “Interactive Text Categorisation: The Geometry of Likelihood Spaces.” Studies in Comput. Intell. 668: 13–34. doi:10.1007/978-3-319-46135-9_2.

———. 2018. “A Study of an Automatic Stopping Strategy for Technologically Assisted Medical Reviews.” In Proc. of Ecir 2018, 672–77. Springer. doi:10.1007/978-3-319-76941-7_61.

Di Nunzio, G. M., F. Beghini, F. Vezzani, and G. Henrot. 2017. “An Interactive Two-Dimensional Approach to Query Aspects Rewriting in Systematic Reviews. IMS Unipd at CLEF eHealth Task 2.” In Working Notes of CLEF 2017. Vol. 1866. CEUR-WS. http://ceur-ws.org/Vol-1866/paper_119.pdf.